ABSTRACT
Background: Multiple choice questions (MCQs) are preferred tools of assessment because of objectivity, ease of scoring and how each MCQ functions as an item can be understood by Item analysis. Aims and Objectives: The aim of the study was (i) to carry out item analysis of MCQs used in formative assessment to know the validity and (ii) to carry out a post validation item analysis of MCQ’s of 1st MBBS students in anatomy to use results for further actions. Materials and Methods: 45 MCQs were administered to 112 students of 1st M.B.B.S. as a formative assessment. Difficulty index and Discrimination index were calculated. Results: Mean difficulty index was 56.67 ± 22.09, and mean discrimination index was 0.35 ± 0.23. Distribution of easy, moderate, and difficult MCQ was 20, 67, and 13%, respectively. About 20% of MCQs were poor, 20% with acceptable discriminating index, 27% had good, and 33% MCQ were of excellent discrimination index. No item was negatively discriminating and all distractor were functional. Very easy and very difficult items had poor discriminating index. Conclusion: Most of items had moderate difficulty and good to excellent discrimination. Too easy and too difficult items showcased poor discrimination, no negatively discriminating item and absence of non-functional distractor suggest good quality framing of the MCQs.
ABSTRACT
Background: Multiple Choice Questions (MCQs) or ‘‘items” are frequently used in medical education to assess students’ knowledge at the end of an academic session. However, item analysis has to be done to confirm their excellence. This piece of work was aimed to analyze test items in the subject of Physiology. In view of the fact that item analysis has never been conducted in our physiology department, thus this study was planned to assess the quality of MCQs and construct a viable question bank. Aim and Objectives: To evaluate the quality of MCQs used so as to develop a pool of valid items to update the question bank. Materials and Methods: Total of one hundred 1st year MBBS students were given 40 MCQs. Analysis was done with the item difficulty and item discrimination indices. Results: Difficulty indices (p) of 14 items were easy while about two MCQ were difficult and the remaining 24 of the items were within acceptable range. The discrimination index (DI) (D) for 40 items; one item showed negative discrimination, ten were having poor discrimination, whereas 11 were having excellent discrimination.Nine items were having acceptable, and nine items were having good discrimination. The DI exhibited slight positive correlation with difficulty index (r = 0.1575 P = 0.05). However, it was not statistically significant. Conclusion: The findings show that most of the items were falling in acceptable range of difficulty and discrimination level. It concludes that Item analysis is a very useful and necessary procedure to obtain valid Items from time to time.
ABSTRACT
Objectives: The main objective of the current study is to examine the items using the item analysis to assess their competency of knowledge in human anatomy. Methods: This is a quantitative descriptive cross-sectional study conducted at the Najran University faculty of Medicine, in the department of anatomy. A 31-second year MBBS students of level three took the multiple-choice question exam comprising 100 questions in one session. Each item was analyzed by item analysis. The planned period of data collection is the first semester of the academic year 2019-2020. The data were analyzed using SPSS version 25, and Excel Software. All analyses were performed using the descriptive frequency, explore and correlation probabilities a p-value, p<0.01 (2-sided) was considered to indicate statistical significance. Results: The assessment of test items use the item analysis that is very difficult question: 25; 25%, difficult question: 8; 8%, average question: 46; 46%, easy questions; 9; 9% and very easy: 9; 9%. The discrimination indices, which are poor discrimination; 34; 34%, acceptable discrimination; 11; 11%,good discrimination; 6; 6%, very good discrimination; 4; 4% and excellent discrimination; 45; 45%. The performance of students in this test showing that a large number of items have a role in distinguishing students knowledge in the anatomy test, but in Simple parts of the scale that the items did not succeed in assessing students. Items of anatomy exams displayed a strong relationship of difficulty (P) and discrimination (D) indices. Conclusion: The MCQs items assess their competency of knowledge in human anatomy. The competence of students in the knowledge of anatomy measured at a high rate of A-type context-rich MCQs can be stem is case scenario, lead-in; question and options; distractors, and key. These items can differentiate good and poor accomplishment students..
ABSTRACT
Background: Multiple choice questions (MCQs) are a common method for formative and summative assessment of medical students. Item analysis enables identifying good MCQs based on difficulty index (DIF I), discrimination index (DI), distracter efficiency (DE). The objective of this study was to assess the quality of MCQs currently in use in pharmacology by item analysis and develop a MCQ bank with quality items.Methods: This cross-sectional study was conducted in 148 second year MBBS students at NKP Salve institute of medical sciences from January 2018 to August 2018. Forty MCQs twenty each from the two term examination of pharmacology were taken for item analysis A correct response to an item was awarded one mark and each incorrect response was awarded zero. Each item was analyzed using Microsoft excel sheet for three parameters such as DIF I, DI, and DE.Results: In present study mean and standard deviation (SD) for Difficulty index (%) Discrimination index (%) and Distractor efficiency (%) were 64.54±19.63, 0.26±0.16 and 66.54±34.59 respectively. Out of 40 items large number of MCQs has acceptable level of DIF (70%) and good in discriminating higher and lower ability students DI (77.5%). Distractor efficiency related to presence of zero or 1 non-functional distrator (NFD) is 80%.Conclusions: The study showed that item analysis is a valid tool to identify quality items which regularly incorporated can help to develop a very useful, valid and a reliable question bank.
ABSTRACT
Background: Assessment is a dominant motivator to direct and drive students learning. Different methods of assessment are used to assess medical knowledge in undergraduate medical education. Multiple choice questions (MCQs) are being used increasingly due to their higher reliability, validity, and ease of scoring. Item analysis enables identifying good MCQs based on difficulty index (DIF I), discrimination index (DI), and distracter efficiency (DE).Methods: Students of second year MBBS appeared in a formative assessment test, that was comprised of 50 “One best response type” MCQs of 50 marks without negative marking. All MCQs were having single stem with four options including, one being correct answer and other three incorrect alternatives (distracter). Three question paper sets were prepared by disorganizing sequence of questions. One of the three paper sets was given to each student to avoid copying from neighboring students. Total 50 MCQs and 150 distracters were analyzed and indices like DIF I, DI, and DE were calculated.Results: Total Score of 87 students ranged from 17 to 48 (out of total 50). Mean for difficulty index (DIF I) (%) was 71.6+19.4. 28% MCQs were average and “recommended” (DIF I 30-70%). Mean for discrimination index (DI) was 0.3+0.17. 16% MCQs were “good” and 50% MCQs were in “excellent” criteria, while rests of the MCQs were “discard/poor” according to DI criteria. Mean for distracter efficiency (DE) (%) was 63.4+33.3. 90% of the items were having DE from 100 to 33%. It was found that MCQs with lower difficulty index (<70) were having higher distracter efficiency (93.8% vs. 6.2%, p=0.004).Conclusions: Item analysis provided necessary data for improvement in question formulation and helped in revising and improving the quality of items and test also. Questions having lower difficulty index (<70) were significantly associated with higher discrimination index (>0.15) and higher distractor efficiency.
ABSTRACT
PURPOSE: This study investigated the relationship between the item response time (iRT) and classic item analysis indicators obtained from computer-based test (CBT) results and deduce students' problem-solving behavior using the relationship. METHODS: We retrospectively analyzed the results of the Comprehensive Basic Medical Sciences Examination conducted for 5 years by a CBT system in Dankook University College of Medicine. iRT is defined as the time spent to answer the question. The discrimination index and the difficulty level were used to analyze the items using classical test theory (CTT). The relationship of iRT and the CTT were investigated using a correlation analysis. An analysis of variance was performed to identify the difference between iRT and difficulty level. A regression analysis was conducted to examine the effect of the difficulty index and discrimination index on iRT. RESULTS: iRT increases with increasing difficulty index, and iRT tends to decrease with increasing discrimination index. The students' effort is increased when they solve difficult items but reduced when they are confronted with items with a high discrimination. The students' test effort represented by iRT was properly maintained when the items have a 'desirable' difficulty and a 'good' discrimination. CONCLUSION: The results of our study show that an adequate degree of item difficulty and discrimination is required to increase students' motivation. It might be inferred that with the combination of CTT and iRT, we can gain insights about the quality of the examination and test behaviors of the students, which can provide us with more powerful tools to improve them.
Subject(s)
Humans , Discrimination, Psychological , Motivation , Reaction Time , Retrospective StudiesABSTRACT
Background: Multiple choice questions (MCQs) are a common method of assessment of medical students. The quality of MCQs is determined by three parameters such as difficulty index (DIF I), discrimination index (DI), and Distractor efficiency (DE). Item analysis is a valuable yet relatively simple procedure, performed after the examination that provides information regarding the reliability and validity of a test item. The objective of this study was to perform an item analysis of MCQs for testing their validity parameters.Methods: 50 items consisting of 150 distractors were selected from the formative exams. A correct response to an item was awarded one mark with no negative marking for incorrect response. Each item was analysed for three parameters such as DIF I, DI, and DE.Results: A total of 50 items consisting of 150 Distractor s were analysed. DIF I of 31 (62%) items were in the acceptable range (DIF I= 30-70%) and 30 had ‘good to excellent’ (DI >0.25). 10 (20%) items were too easy and 9 (18%) items were too difficult (DIF I <30%). There were 4 items with 6 non-functional Distractor s (NFDs), while the rest 46 items did not have any NFDs.Conclusions: Item analysis is a valuable tool as it helps us to retain the valuable MCQs and discard or modify the items which are not useful. It also helps in increasing our skills in test construction and identifies the specific areas of course content which need greater emphasis or clarity.
ABSTRACT
Background: In medical education, multiple choice questions/Items are the more frequently used assessment tools to assess the knowledge abilities and skills of medical students, for being their objectivity, wide coverage in less time. However only the Quality Items gives a valid and reliable assessment. The quality of an Item is determined by difficulty index (DIF I), Discrimination Index (DI) and Distractor efficiency (DE). Aim of the study was to know the quality of Items in pharmacology by Item analysis and to develop a MCQs bank with quality Items.Methods: The present study was conducted on 150 II MBBS students of Guntur Medical College, AP, India. A class test containing 50 Items with 150 distractors from topic chemotherapy was conducted. Item with the correct choice/response was awarded with one mark and with the wrong choice zero marks, no negative marks. Each test Item was analysed with DIF I, DI and DE and the results were tabulated and tested statistically, with unpaired "t" test.Results: Mean DIF I, DI, DE values with standard deviations in the present study are 44.72+17.63%, 0.30+0.12%, 84.48+24.65 respectively. DIF I of 32 (64%) items was good to excellent range (31%-60%) 9 (18%) Items were easy (>61%) and 9(18%) Items were difficult (>30%). DI of 10 (20%) Items was good (0.15 to 0.24.) 29 (58%) Items were excellent with DI > 0.25 and 11 (22%) Items were poor with DI <0.15. Among 150 distractors, 127 (85%) were functional distractors (FDs) and 23 (15%) were non-functional distractors (NFDs). DE of 33 (66%) items with nil NFDs was 100%, for 12 (24%) Items with one NFD, was 66.6%, for 4 (8%) items with 2 NFDs was 33.3% and for 1 (2%) Item with 3NFDs DE was 0%. When unpaired "t" test was applied to the means of "difficult" and "easy" Items, 96.22+11.33% SD, 51.44+29.31% SD respectively, the p-value obtained was 0.00058, which was highly significant.Conclusions: The study showed that Item analysis is a valid tool to identify quality Items, which assess, the students� knowledge abilities and discriminate different levels of performance abilities of students effectively.
ABSTRACT
Background: The accurate, reliable and timely assessment of students is an essential domain of teaching duringMedical professional courses. The Multiple Choice Questions (MCQ) are time tested method of ready assessmentof undergraduate students. Although it evaluates student’s cognitive knowledge but does not evaluate professionalskills. However it is said that MCQs emphasize recall of factual information rather than conceptual understandingand interpretation of concepts.Objectives: The main objective of the study is to analyse the items with the help of item analysis and select theitems which are good for incorporation into future question bank with reliability.Materials and Methods: This study was done in Department of Anatomy, AIIMS, Patna. A 396 first year MBBSstudents of different batches took the MCQ test comprising 60 questions in two sessions. During the evaluationprocess of MCQ’s each correct response was awarded one mark and no marks was awarded for any incorrectresponse. Each item was analysed for difficulty index, discrimination index and distractor effectiveness.Results: The overall mean of Facilitative value, Discrimination Index, Distractor Effectiveness and CorrelationCoefficient was 66.09 (±21.55), 0.26 (±0.16), 18.84 (±10.45) and 0.55±0.22 respectively.Conclusion: The framing of MCQ should be according to Bloom’s classification to assess cognitive, affective aswell as psychomotor domain of the students. The MCQ having poor and negative discrimination should bereframed and again should be analysed.
ABSTRACT
Introduction@#The clinical skills training at medical schools provides the opportunity for future medical doctors to deal with the client with proper care, diagnosis of the disease, first aid, treatment, nursing, treatment, counseling to address the complexity of the problem solving and the ethical attitude of the doctor. To achieve this objective, it is necessary to assess the level of knowledge, skills and attitudes students have acquired.@*Goal@#To analyze assignment of basic clinical skills assessment and to identify the level of кknowledge and skills students who have graduated second year medical program at “Ach” Medical University during 2016- 2017 academic year.@*Materials and Methods@#The study was used as a descriptive model to measure the reliability of the assignment, the difficulty factor of tasks, and the Hoffsten’s scores based on the tasks and performance of each station and compared with the indicators.@*Results@#Based on Hoffsten’s study on the success rate of examiners at the 5 stations, the Hoffsten’s score level of clinical examination was 68 percent, the physical examination station was 64 percent, the station’s diagnostic level was 71 percent, the laboratory was 70 percent and the nursing station was 70 percent.@*Conclusion@#At each clinical trial, the differential diagnosis of each individual clinical trial, clinical interview, nursing station and visual diagnostic station (DF> 95), at the laboratory and at the physical examination station, assess the student with a higher grade of difficulty factor (DF> 80) to the Hoffsten’s score of the basic clinical skills exam is set to be 70 percent.
ABSTRACT
Introduction@#One of the quality assurance measurements for medical schools is the achievement of students who have graduated in the assessment of the knowledge, skills and attitudes they are trained in.@*Goal@#To analyze assignment of theoretical and practical exam and to identify the level of кknowledge students who have graduate at “Ach” Medical University during 2015-2016 academic year.@*Materials and Methods@#The study was conducted on a cross sectional and descriptive study through the based on the task of analyzing the 261 graduate students theoretical and practical exam performance of the bachelor degree in Medicine, Dentistry, Traditional Medicine and Nursing of Ach Medical University of Mongolia /AMU/ and was assessed and to identify a reliability coefficient, difficulty factor, discrimination index, Hoffsten’s score. @*Results@#The reliabiliy coefficient of graduate exam meets requirement when it’s 0.94-0.96. According to the analysis of the 300 test of the each classroom of graduates was 70 percent (n=202) with weak dicrimination index, difficallty factor was more than 50 percent too easy, The Hoffsten’s score to which exam was passed of Medical graduates is 70 percent, traditional medicine is 87 percent, dentistry is 79 percent, the nursing is a Hoffsten’s score was 80 percent.@*Conclusions @#The reliability coefficient the theoretical exam of the graduates’ knowledge is convenient for all occupations, and whole field examines the weak difficulty index (DI≤0) for all field examinations. The Hoffsten’s score is 70% above the medical field. Graduate assignments can not discriminate graduates’ knowledge and skills levels and the difficulty factor graduate examination was very easy.
ABSTRACT
Objective In a large-scale clinical medical examination, A2 type best multiple-choice test questions to the same knowledge were used respectively in simple and complex form, to compare the difficulty and discrimination indices of the two forms of test questions and provide evidence to the improve-ment of clinical medical examination. Method In a large-scale clinical medical examination more than 4000 candidates participated, and 20 questions to different knowledge points were randomly selected and used in the examination respectively in simple and complex A2 type best multiple-choice test questions. The difficulty and discrimination indices of the two forms of test questions were compared. Results The average difficulty coefficient of the 20 simple test questions (65.5 words per question in average) is 0.6829, and the average discriminative powers are 0.2675 and 0.2579 respectively using identification index method and point biserial correlation method. The average difficulty coefficient of the 20 complex test ques-tions (135.5 words per question in average) is 0.7095, and the average discriminative powers are 0.3065 and 0.2967 respectively using identification index method and point biserial correlation method. Conclusion To the same knowledge points, the average difficulty of complex A2 type best multiple-choice test questions is slightly lower than the simple ones, while the average discriminative power is increased in the complex questions. The complex A2 type test questions are of higher quality and more in line with the requirements of the medical entrance examination, medical education and examination reform.
ABSTRACT
Objective: To establish a method of quantitative analysis of multi-components by single marker (QAMS) for medicinal materials and pieces of Cornus officinalis. This method was used in combination with electronic-eye and electronic-tongue technique, and the best steaming time of Cornus officinalis was selected. Methods: Medicinal materials and pieces of C. officinalis were used as the research objects. The contents of five components were determined by establishing the relative correction factor (RCF) of gallic acid, 5-hydroxymethyl furfural (5-HMF), morroniside, cornuside, and internal reference loganin in C. officinalis. Color and taste were measured by electronic eye and electronic tongue technique. The data were analyzed by principal component analysis (PCA), and the best steaming time was optimized by analyzing the results of three methods. Results: The five compounds were well separated. The RSD values of precision and reproducibility were all less than 2%. The stability was good in 24 h. The linear relationship among the concentration and peak areas of the five compounds was all linear (r ≥ 0.999 6). The average recoveries were between 98% and 100.1% and the RSD values were all less than 2%; The RCFs of loganin with the other four compounds were 0.560, 1.344, 1.255, and 0.972 in a linear range. In the principal component analysis (PCA), the sums of main components were 94.618% and 94.98% and the discrimination indexes (DI) were 98 and 93, which indicated that all the samples of C. officinalis could be distinguished well by the electronic-eye and the electronic-tongue. The results showed that the optimum steaming time of C. officinalis was 4 h. Conclusion: The best steaming time of C. officinalis can be optimized by the combination of QAMS with electronic-eye and electronic-tongue techniques.
ABSTRACT
Objective Evaluating the effect of situational questions for qualifying the students'abilities in physiolo-gy examination .Methods Comparing the difficulty coefficients and discrimination indexes between situational and traditional choice questions .Results Compared with the traditional choice questions , the difficulty coefficient of situational questions increased , while the discrimination indexes were more reasonable .The discrimination indexes of situational understanding questions were higher than those of the traditional memory and understanding questions . There were no difference between discrimination indexes of the situational application questions and those of tradi -tional application questions .Conclusions Situational questions not only improved the quality of examinations , but also facilitate evaluating students'learning ability .
ABSTRACT
Multiple-choice question as one best answer (OBA) is considered as a more effective tool to test higher order thinking for its reliability and validity compared to objective test (multiple true and false) items. However, to determine quality of OBA questions it needs item analysis for difficulty index (PI) and discrimination index (DI) as well as distractor efficiency (DE) with functional distractor (FD) and non-functional distractor (NFD). However, any flaw in item structuring should not be allowed to affect students’ performance due to the error of measurement. Standard error of measurement (SEM) to calculate a band of score can be utilized to reduce the impact of error in assessment. Present study evaluates the quality of 30 items OBA administered in professional II examination to apply the corrective measures and produce quality items for the question bank. The mean (SD) of 30 items OBA = 61.11 (7.495) and the reliability (internal consistency) as Cronbach’s alpha = 0.447. Out of 30 OBA items 11(36.66%) with PI = 0.31-0.60 and 12 items (40.00%) with DI = ≥0.19 were placed in category to retain item in question bank, 6 items (20.00%) in category to revise items with DI ≤0.19 and remaining 12 items (40.00%) in category to discard items for either with a poor or with negative DI. Out of a total 120 distractors, the non-functional distractors (NFD) were 63 (52.5%) and functional distracters were 57 (47.5%). 28 items (93.33%) were found to contain 1- 4 NFD and only 2 (6.66%) items were without any NFD. Distracter efficiency (DE) result of 28 items with NDF and only 2 items without NDF showed 7 items each with 1 NFD (75% DE) and 4 NFD (0% DE), 10 items with 2 NFD (50% DE) and 4 items with 3 NFD (25% DE). Standard error of measurement (SEM) calculated for OBA has been ± 5.51 and considering the borderline cut-off point set at ≥45%, a band score within 1 SD (68%) is generated for OBA. The high frequency of difficult or easy items and moderate to poor discrimination suggest the need of items corrective measure. Increased number of NFD and low DE in this study indicates difficulty of teaching faculty in developing plausible distractors for OBA question. Standard error of measurement (SEM) should be utilized to calculate a band of score to make logical decision on pass or fail of borderline students.
ABSTRACT
BackgroundHealth professional licensing was introduced in Mongolia in 1999. Medical school graduates shouldpass the health professional licensing exam (HPLE) to be registered. It was informed that HPLEsuccess rate has been decreased for last few years among graduates who passed final theoreticexam (FTE). There has been no research conducted to explain the reasons of such trend. Thisresearch aims to conduct a comparative assessment of MSQs used for both HPLE and FTE.GoalTo analyze examination and test to identify the level of medical knowledge of students who graduateas medical doctor at “Ach” Medical University during 2011- 2015.Materials and MethodsThis is a cross sectional descriptive study. it employed a statistical analysis of 2950 MSQs (24version) that were used for the HPLE by the Health Development Center of the MOH (N=16)and FTE by the “Ach” Medical University (N=8) between 2011 and 2015. Test sheets of HPLE(N=728) and FTE (N=686) were assessed in order to identify a reliability of tests, difficulty index,discrimination index using QuickSCORE II program of the test reading machine with a mode of“Scantron ES-2010”.ResultsThe success rate was much higher in FTE than it in HPLE between 2011 and 2015. The successrate of HPLE decreased dramatically starting from 2013 (87%) to 2014 (4%) and 2015 (24%) whilethe same rate of FTE was stable and almost 100%.FTE’s reliability coefficient of 2011-2015 years meets requirement when it’s 0.92-0.96. HPLE’sreliability coefficient of 2013 and 2014 years don’t meet requirement.From all of the MCQs that has been used in FTE‘s 97% and in HPLE’s 80% are positive discriminationindex which means possible to identify medical school graduates knowledge.ConclusionOur findings confirmed that the success rates of HPLE among medical school graduates are beingquite low.Reliability coefficient of HPLE tests were less reliable (КР20=0.66-0.86) than FTE (КР20=0,92-0.96) and particularly tests for 2014 and 2015 were more difficult and were with high percentage ofnegative discrimination.Test score between HPLE and FTE of 2011-2015 is direct linear correlation.
ABSTRACT
Background: The most common tool used for assessment of knowledge is the essay questions. Their evaluations depend upon test and item analysis which is consisting of analysis of individual questions and analysis of the whole test. The objective of our study was to calculate the Item analysis (Facility value, FV and discrimination index, DI) of questions given in terminal examination of MBBS students and observes the adequacy of questions framing in examination. Methods: Study contains 150 medical students undergone terminal examination consist of questions of essay type, structured essay type and short answer questions were given to the students. We divide them into high ability group and low ability group. Mark range was decided for essay type, structured essay type and short answer type questions and FV & DI was calculated. Results: Studies described that facility values of 62.5 percentage questions were come under recommended & acceptable range and 50 percentage questions were come under acceptable range. Discrimination value of 100 percentage questions were come under recommended & acceptable range where 75 percentage questions were come under acceptable range. Conclusion: The importance of item analysis is highlighted from our results. For improvement of examination items with average difficulty and high discrimination should be implemented into future examinations to improve test scores and properly discriminate among the students.
ABSTRACT
Introduction: Difficulty index (P) and discrimination index (D) are the parameters used to evaluate the standard of multiple choice questions (MCQs) in examination. Accordingly the standard of MCQs can be interpreted as excellent, acceptable or poor. This study was intended to find out the standard of MCQs in formative examination in Physiology. The study also intended to find out correlation between P and D. Materials and Methods: There were 240 MCQ items, taken from the past 4 year batches of 100 students and were analyzed for level of difficulty and discrimination index. The relationship between them for each test item was determined by Pearson correlation analysis using SPSS 11.5. Results: There was a wide distribution of item difficulty indices (8.57 to 95.71) and discrimination indices (-0.54 to 0.8).The mean difficulty index (P) was 52.53 + 20.59 and mean discrimination index was 0.30+ 0.18. On average, about 23% of the MCQ items were easy (P >70%), while about 15% were difficult (P <30%). The remaining 62% items were within acceptable range (30 to 70%). In all 4% of the items showed negative discrimination and 21% of the items exhibited poor discrimination. The remaining 75% of the items were in the range of acceptable to excellent discrimination. The discrimination index exhibited slight positive correlation with difficulty index (r = 0.191, P=0.003<0.01). The maximal discrimination (D=0.6-0.8) was observed with moderately easy/difficult items (P = 40% - 60%). Conclusion: The majority (75%) of the items was acceptable as far as difficulty and discriminative indices were concerned. Moderately easy/difficult items had maximal discriminative ability. Too easy and too difficult items gave poor discrimination index. Negative discrimination was observed in only 4% of the items indicating faulty items or incorrect keys.
ABSTRACT
A new method for quality control of multiple-choice tests was developed with the assumption that there are two classes of items-beneficial and nonbeneficial-to evaluate students' ability. The information function is applied to distinguish these two classes. Of 316 items on multiple-choice tests at Tokai Medical School in 1993, 224 (70.9%) were beneficial and 92 (19.1%) were nonbeneficial. Between these classes, the averages of passing percentage and the discrimination index were significantly different. The percentage of beneficial items varies with the form of items, whereas the average discrimination index does not.
ABSTRACT
PURPOSE: In 2002, extended-matching type (R-type) items were introduced to the Korean Medical Licensing Examination. To evaluate the usability of R-type items, the results of the Korean Medical Licensing Examination in 2002 and 2003 were analyzed based on item types and knowledge levels. METHODS: Item parameters, such as difficulty and discrimination indexes, were calculated using the classical test theory. The item parameters were compared across three item types and three knowledge levels. RESULTS: The values of R-type item parameters were higher than those of A- or K-type items. There was no significant difference in item parameters according to knowledge level, including recall, interpretation, and problem solving. The reliability of R-type items exceeded 0.99. With the R-type, an increasing number in correct answers was associated with a decreasing difficulty index. CONCLUSION: The introduction of R-type items is favorable from the perspective of item parameters. However, an increase in the number of correct answers in pick 'n'-type questions results in the items being more difficult to solve.